High Dynamic Range Imaging at the Quantum Limit with SPAD-based Image Sensors

This paper examines methods to best exploit the High Dynamic Range (HDR) of the single photon avalanche diode (SPAD) in a high fill-factor HDR photon counting pixel that is scalable to megapixel arrays. The proposed method combines multi-exposure HDR with temporal oversampling in-pixel. We present a silicon demonstration IC with 96 × 40 array of 8.25 µm pitch 66% fill-factor SPAD-based pixels achieving >100 dB dynamic range with 3 back-to-back exposures (short, mid, long). Each pixel sums 15 bit-planes or binary field images internally to constitute one frame providing 3.75× data compression, hence the 1k frames per second (FPS) output off-chip represents 45,000 individual field images per second on chip. Two future projections of this work are described: scaling SPAD-based image sensors to HDR 1 MPixel formats and shrinking the pixel pitch to 1–3 µm

Smart wide-field fluorescence lifetime imaging system with CMOS single-photon avalanche diode arrays

Wide-field fluorescence lifetime imaging (FLIM) is a promising technique for biomedical and clinic applications. Integrating with CMOS single-photon avalanche diode (SPAD) sensor arrays can lead to cheaper and portable real-time FLIM systems. However, the FLIM data obtained by such sensor systems often have sophisticated noise features. There is still a lack of fast tools to recover lifetime parameters from highly noise-corrupted fluorescence signals efficiently. This paper proposes a smart wide-field FLIM system containing a 192×128 COMS SPAD sensor and a field-programmable gate array (FPGA) embedded deep learning (DL) FLIM processor. The processor adopts a hardware-friendly and light-weighted neural network for fluorescence lifetime analysis, showing the advantages of high accuracy against noise, fast speed, and low power consumption. Experimental results demonstrate the proposed system's superior and robust performances, promising for many FLIM applications such as FLIM-guided clinical surgeries, cancer diagnosis, and biomedical imagin

A 192×128 Time Correlated SPAD Image Sensor in 40-nm CMOS Technology

A 192 X 128 pixel single photon avalanche diode (SPAD) time-resolved single photon counting (TCSPC) image sensor is implemented in STMicroelectronics 40-nm CMOS technology. The 13% fill factor, 18.4\,\,\mu \text {m} \times 9.2\,\,\mu \text{m} pixel contains a 33-ps resolution, 135-ns full scale, 12-bit time-to-digital converter (TDC) with 0.9-LSB differential and 5.64-LSB integral nonlinearity (DNL/INL). The sensor achieves a mean 219-ps full-width half-maximum (FWHM) impulse response function (IRF) and is operable at up to 18.6 kframes/s through 64 parallelized serial outputs. Cylindrical microlenses with a concentration factor of 3.25 increase the fill factor to 42%. The median dark count rate (DCR) is 25 Hz at 1.5-V excess bias. A digital calibration scheme integrated into a column of the imager allows off-chip digital process, voltage, and temperature (PVT) compensation of every frame on the fly. Fluorescence lifetime imaging microscopy (FLIM) results are presented

Motion-triggered 3D imaging with a Time of Flight CMOS SPAD image sensor

Time of flight (ToF) imaging has become a prime 3D sensing technology in consumer and industrial applications, favoured for its low mechanical and computational complexity amongst ranging techniques. Despite its advantages, ToF sensors exhibit high data rate and power due to their temporal resolution and fast frame rates. In Internet of Things (IoT) networks, energy and output data efficiency are crucial for sensors to prevent bandwidth saturation of the processing nodes, while scaling the number of connected devices and extending the life of systems on limited energy supplies. 3D imaging applications, such as surveillance, object tracking and classification in industrial robotic automation, feature conditions where the event interval and frequency of events can be far lower than the time spent by the sensor capturing static scenes. ToF cameras lack the embedded intelligence to recognise frames of interest. Temporal contrast vision sensors, on the other hand, efficiently limit the expense of energy and data bandwidth to detect motion by outputting low data rate binary frames. However, while these sensors integrate well with standard intensity imaging, they do not provide 3D image sensing functionality. The goal of this research is to investigate the feasibility of combining vision algorithms with ToF imaging topologies to design a motion-triggered 3D image sensor. A frame differencing vision architecture is leveraged to limit ToF acquisition to short event-driven duty-cycles, reducing the high consumption and data traffic to external processing and shortening the active time of the ToF emitter. The design strategy is founded on hardware resource sharing in the implementation of both motion detection and ranging functions to minimise pixel area scaling. The shot noise-limited light detection provided by single photon avalanche diodes (SPAD), further reinforced by their recent integration in advanced digital CMOS technologies, makes this detector the enabling factor for the design of the first fully digital motion-triggered 3D image sensor, superior in motion contrast sensitivity and noise performance to analogue-based vision sensors. A 128×128 SPAD motion-triggered 3D image sensor was designed in STMicroelectronics 40 nm CMOS process. 40 µm×20 µm pixels integrate two 16-bit time-gated counters to acquire a 6-bin ToF histogram. The counters are repurposed for acquisition of consecutive frames for motion detection by a frame comparison algorithm. An embedded column-parallel processor performs relative frame differencing with an adaptive per pixel threshold. The sensor achieves a 78% power saving and a data rate reduction of 10 times when operating the sensor in motion-triggered 3D imaging at 30 fps, compared to continuous ToF acquisition. High motion sensitivity below 4% contrast is achieved with a 1.39% pixel-to-pixel contrast FPN by shot noise-limited integration and spatial noise filtering over a dynamic range exceeding 60 dB. ToF 3D imaging is accomplished by a 100 fps histogram acquisition triggered by motion in the scene with 1.5 cm depth accuracy over a measured 3 m range

Real-time fluorescence lifetime actuation for cell sorting using a CMOS SPAD silicon photomultiplier

Time-correlated single photon counting (TCSPC) is a fundamental fluorescence lifetime measurement technique offering high signal to noise ratio (SNR). However, its requirement for complex software algorithms for histogram processing restricts throughput in flow cytometers and prevents on-the-fly sorting of cells. We present a single-point digital Silicon Photomultiplier (SiPM) detector accomplishing real-time fluorescence lifetime-activated actuation targeting cell sorting applications in flow cytometry. The sensor also achieves burst-integrated fluorescence lifetime (BIFL) detection by TCSPC. The SiPM is a single-chip complementary metal oxide semiconductor (CMOS) sensor employing a 3232 single-photon avalanche diode (SPAD) array and 8 pairs of time-interleaved time to digital converters (TI-TDCs) with a 50 ps minimum timing resolution. The sensor’s pile-up resistant embedded centre of mass method (CMM) processor accomplishes lowlatency measurement and thresholding of fluorescence lifetime. A digital control signal is generated with a 16.6 s latency for cell sorter actuation allowing a maximum cell throughput of 60,000 cells per second and an error rate of 0.6%